Exploratory Data Analysis (EDA) - Sample
1. Department of Housing Preservation and Development (HPD)¶
Introduction¶
The Department of Housing Preservation and Development (HPD) reports on buildings, units, and projects to support the "Housing New York Plan", and to assist homeowners or special populations. HPD monitoring data is provided in two datasets "Housing New York Units by Building" & "Housing New York Units by Project" and published quarterly. Both files contain project's related information for "specific housing development or preservation project". Data granularity includes and is not limited to: Building ID, House Number, Borough, Building Identification Number (BIN), etc.; however, these reports do exclude types of projects designed to assist homeowners or special populations and may are marked “CONFIDENTIAL”.
2. Data Quality Objective (Sample)¶
2.1 Housing New York Units by Building - Monthly Report (Order to Repair / Vacate Orders)¶
2.1.1 Introduction¶
Understanding and ensuring transparency in property maintenance activity are key elements for monitoring New York City's risk exposure inherent from project's financial overruns, while providing safe living conditions for its residents. The objective is to set standards and metrics for monitoring business progress through the analysis of building conditions around New York City and its Boroughs.
The success of the strategy depends on stakeholder contributions, along with integrated activities connecting field-data to internal systems; provided, that field-information can be corroborated and calculated-metrics reproducible.
The following information provides a monthly overview of activities aiming to briefly analyze and portray housing activity around New York's 5 Boroughs. Included here in are a set of brief statistics providing monthly and year over year observations with a semi-interactive graphical interface.
The report contains the following:
- Monthly highlights of business activity
- Month over Month and Year over Year Comparisons
- Interactive Visualizations
- Limitations
- Data Quality Information and General Statistical measures
Note: The information contained in this document uses public available data provided by NYC OpenData.
3. Interactive Visualizations¶
3.1 Usage Instructions¶
Visualizations available in the document are fully interactive. Users can Turn-On and Turn-Off desired variables with a click of a mouse, hover over the plot to view details of the variables as well as Zoom-in or Zoom-out selected information pressing hold + left-click with the mouse of the desired area. Hovering over the plotting area will reveal additional features for the plot displayed on the upper right-hand side corner.
4. Data Analysis and Observations¶
4.1 Overview¶
On average ~33 units have been vacated monthly throughout the 5 Boroughs where 79% or 357 units have been vacated due to Fire Damage. Overall, the trend continues a predicted path in the number of vacated due to Fire Damage and Illegal Occupancy when compared to previous year. On average the largest number of vacated units due occur during the first months of the year with a slight decrease in activity on the months of May and June. Accounting for the impact of the Pandemic on number of Total Vacated Units, we can only account for a modest decline of -6.4%.
Year to date there are 512 vacated units across the 5 Boroughs, with the Bronx and Brooklyn accounting for 68% of the total. The Bronx has the highest total vacated units ratio due to Fire Damage at 89%. Brooklyn on the other hand has the highest number of vacated units due to Illegal Occupancy (84) accounting for 49% of its total vacated units; trailing Manhattan (87) for the third highest number of vacated units due to Fire Damage at 82. </hr>
### 4.2 Highlights * An outlier was detected in the Borough of Queens with a **"Vacate Effective Date" of: 2040-08-19** | **2 units in Building ID: 424585** due to Illegal Occupancy * YoY the number of vacated units due to Illegal Occupancy has declined -34%, and -31% due to Fire Damage respectively * Fire Damage and Illegal Occupancy continue to be the main drivers accounting for 96% of all vacated units * The Total number of vacated units has declined -35% when compared to the same period on previous year * In Brooklyn the number of vacated units due to Illegal Occupancy increased by 20% * Compared to previous month vacated units due to Fire Damage increased 38%
- Keys:
YoY = Year Over Year ~ = Approximation
5. Limitations of Analysis and Functionality¶
The document is intended to provide simple, yet objective description of the data observed. This brief provides a preliminary overview of public available data, and limitations to the analysis should be noted. It is important to understand that limitations represent weaknesses of the research, data, design and other characteristics that may impact / influence outcomes and conclusions of the analysis. Some of the limitations may include statistical depth, limited variables, quantitative / qualitative review and plotting visuals.
6. General Dataset Statistics¶
The following provides general statistics about the dataset, including correlations, distributions, variable inspection details among other metrics with some interactivity.